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UNEARLY UNRECOGNIZABLE PATTERNS 


Inirodiiclion. The central theme of this study :s the classification of 
certain jpeornttrical properties according. to the type of computation 
necessary to determine whether * given ftfc'Urc has them, Consider, for 
srample, the InJlnwing algorithm to determine whether a figure X is 
n?ru¥s. For each pair of points (p.f) we define the function 

ip,! X 1 — 1 if (p E X and q E X and midpoint (p, q'i £ X) 


= t) Othei-w:^;,. 


Then X is convex if and only if no d w (,X) = 1 for arty pair of points 
This -shows, in a sense we shall presently define more precisely, that 
the "global” property of eonveiity can be determined by a simple compu¬ 
tation from the “loeflk” properties (l^,. 2 Thus, if means that 

“X is convex." we have 


# X- < 1 - 


Now we generaliae this. We «iy that a p-mp*rty ip is of order £ if k is the 
smallest integer for which there exist* a family 4 of predicates each of 
which depends only on a subset of k points of the figure X r and real 
numbers undated with each member <S> of >t auc-h that 



In this sense, we can assert that Ovr order of i& Of (WMf it. The de- 

1 armination of the orders of simple geometrical properties turns out to 
he far from trivial and presents many surprises. In fact, the greater part 
of the following analysis scorns to be needed to prove that con naeLfldW.w 
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is not of any finite- order, i,e,. 
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ns ii no k far EiLfilcft ernddneat itf c/ order lb. 

This remains true if we relax the definitions, as we shall, to make the 
sums finite by considering the pJaiie 6s a fine-grained infinite chess-board, 
considering its squares lo be points, and allowing only figures which contain 
ouch square entirely or not at aJ] — that ia, wo will consider a discrete 
model of geometry. 

Apart fnom the purely mithnutietl interest of the result* that come 
from it. we consider Ibc concept of fin clt order worthy of study for a number 
of reasons connected with the theories of computation and of pattern 
recognition, We shall briefly outline some of these reasons. 

Afafj'iHfisfl of This Study. 

(a> If cm 1 cs. global geometric properties. In problems of geometric pattern 
recognition, one is led to a**,: to what exient can one use “local" properties— 
evidence obtained from looking «t small portions of on object—as a basis 
for judgement* about the "global" character of the object. For example, 
one can distinguish "line'’ drawings front other pictures on the basis of the 
existence of no interior points in the drawing—and this can be determined 
by a simple Combination of evidence obtained from examining arbitrary 
Email neighborhoods. On the other hand, one cannot obtain "local" evidence 
in favor of a drawing being " connected”— or so one might suspect- without 
having to combine such collections of evidence by a very complicated 
procedure. 

Our first attempt to study this was based on ibc idea of dianiete^-raflthickvi 
predicates, i,e,, the restriction On the "local" properties ls on the diameter 
cf the set of points on which they depend rather than the number of points. 
The results of this study are sumnuiriKd in glX. However it soon became 
Clear that the more interesting concept is order-restriction, and that the 
distinction we were seeking was, not so much a question of geometry as a 
question about the theory of computation. 

(hi ^serial .as parallel compulation. What characterizes the extent to 
which an alftOrilhm can have an essentially serial r as oppoEed to parallel, 
character? 3 hat is, to what degree can $ computation he sped-up by doing 
several subcomputatmm a I the same time':' One would suspect, for example, 
that in many successive approximation computations there is littie to be 
gained except, ut great, expense and redundancy, by parallel processing. 
Vic were led to suppogt that the same ift true for geometric Connectedness 
recognition. One way to recognize that a act Ls disconnected (connected) 
is to find that there Ls a (no) curve dividing the ssel without- intersecting it, 
Om? could therefore examine in parallel ati possible separating curves. 
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rdLh.tri- than $eri-ully iraw through the paths within the ^et. Rut U would 
seeai that the phCt of Bpwding-up the computation this way is superbly 
costly, and one kwhs for A way to pvt theoretical estimates- of whst i& tire 
eichange rate between the minima] serial and parallel amounts of compU’ 
tat-ion. (The goal must be an exchange-cost curve.) One might hope that 
study of a particular problem, e.g., connectedness, would yield some insight 
into this general question of computation complexity for finite problems 
comparable wLdti r say, that achieved in the theory of complexity of the 
recursive functions (Blum)- 

(c) Theory of perceptTfuis and linear separability. The pattern-recognition 
scheme known as the perceptivn (Rosenblatt jfij) is known to he capable 
of lewming to make any pattern discrimination which is- within the scope 
of its potential ability — that is* if there is a set of parameter valuta that 
will suffice;, it will find therm Thus* a good deal ia known ahemt this system's 
learning ability, and therefore one is particularly interested to know what 
Ls the scope of potential ability. Curiously enoughs there seems to he nothing 
in the Large perccptrcn literature on this question., At'vd the present paper 
HOH to he tlie first to link the linear-separation problem with the (EOfiJaiOtrifr 
property problem. 

The perception (and its derivatives) are of considerable interest, mathe¬ 
matically because they are perhaps the simplest nontrivial parallel 
machines. One therefore ought to Understand them thoroughly—as a Sort 
of "linear case' 1 —if one is to get any satisfactory theory of "higher-order" 
parallel computation schemes. 

(4) iVfstAemaftftii MpceJe. Linear separation computations have con¬ 
siderable mathematical significance In themselves. For example* if we 
ask for a maximum likelihood decision process based on Bayesian use of 
the results of statistically independent experiments, one obtains (Minsky 
and SelfrLdge |Sj] a Jinear aepsration procedure. For another example, the 
grrWTahaAtlon (in $1) of BooLean disjunctive normal form appeals to yield 
surprising and fruitful results, Finally, the combination of group theory and 
linear inequalities seem* to promise some new combinatorial techniques. 

1. Theory of linear Rouleau reparation functions. Jn this section we shall 
confine ourselves to the analyei* of the linear representation of predicates 
defined on an abstract set R , without any additional mathematical structure. 
1'he theorem* proved here will be applied in later sections to eels with 
geometrical or tafMfaftoal structures, When necessary fur truth ft must 
be taken as finite. 

Our theory deals with predicates defined on subsets of a given base 
space which we shall consistently denote by R. We use the following no- 
tational conventions; 
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■W Let R he an arbitrary eet and 1? a family of subset* of R. Using 
the letters X„ Y r Z, foe subsets of R it is natural to associate with Jp 
a predicate &<t(X} which ie TRUE if and only if X£E 

{ii) We shall y&c the letters £ and if> to denote predicates defined on the 
set of subsets of R r 

W'e ifhal] use- the notation tf(X) sometime* to mean the predicate whose 
value for a given X is TRUE or FALSE, sometimes to mean a binary act 
function whose value La 1 or 0. When we wish to employ che two setuses 
in the same contest we adapt the notation P^tXr| for the binary function 
whom' value ie I it>(X> is TRUE and ft if f(X) is E4LHE, We will usually 
use this only when there is a possibility of ambiguity, to distinguish 
between pi < &1 - 1, which ie true, and 3 < f& = 1~\ : . which ■» false, 
(iii> Occasionally it will be convenient in examples to use the traditional 
representation of ^(X) as a function of ft ‘‘liuolssn variables" where 
ft — |if |. If the elements of R are Xj, it La traditional to thusfe. of a 

SFubfiet X of ii as an assign* lion of the values, \ or U to Xi according to whether 
the point *, is in X or not, i.e., <L *;” is uaed ambiguously to stand for the 
rth point in the given enumeration of R, and for the set function p, £= X - ], 
Tilts nutation is particularly convenient when is represented in the 
form of a standard Boolean function of two variables, Thus X , Vx, is a 
way of writing the set function 

- Ti.EJt or 

(iv) We need to express the idea that a function may depend only on 
e wheel of the points of /?. We denote by S(tf) the smallest subset S of 
K with the property that, for any *ub**t X, 

MX) =MX(~)S) r 
We call j?lol the ffnpjjijrf of J. 

(v) Let be a set of hinary set-fij notions on R. We say that ^ ia a fincar 
threshold function with rerpeer as * if to each member .y, of $ there corresponds 
a real number a, such that, for some reel number fl: 

4>[X) — FI 
*£+ 

This la oftra written mohe briefly 

We denote by L(+) the set of functions expressible in this way, 

Evi) We now introduce the central concept of order. The order of 4> is 
the smallest k for which there is a # satisfying 
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where | £’■(£■) j ic the C4ftrdlanHlLly of 5 (^J, 

Functions of order 1 appear Ltl the literature under the name of “linear 
threshold functions." It should be noted that the order of a constant 
function Ls zero, hence the number f? in the definition of Lt» can be re¬ 
placed by U (or any other numbed without changing the definition of 
order. Mute also that the definition is unchanged if wo use dr 

instead of I. assuming. when necessary, that R is finite!. 

£vii) 4 is called a nssf; if there is a set A such 1 liar. 

} = j X ] <; 

W e denote thus fu net ion by <p A . 

In point-function notation a mask is a function of the iormi 

yiAyt A'”Ay ( 

where j y, \ is the subset A of if. In particular cooctant functions are masks. 
Uniter Representation. 

PfiorosiTio?!. AU ttHWfcs are of order 1 . 

Proof. For each 1 0 A define #,UC) as !"*E^ 7 - Then 

T £ 4>r'£ \A\~\, 

3-E * 

Jn p-wretcular the functions ^ and + T *re of Otder 1- Similarly the functions 
jrVy, jcA y, xZ)y arc of order I. But the L| exclusive or," * -gjy, and its 
complement, x. = y, are of order £, 

Example (i). x t \f x t y is of order V- 

pc L + x s + X.j > (Tj, 

* i A *5 A J] is also of order 1 j 

r~ji -i- 4- Aj ^ 2~[ r 

j|jfj ■= | r L -I- j| — tj > 1 - |""ki — x 7 > ti | is of order L 
V -ii = F-^i + {1 — * j) > OH = r*!! - xi > — I - !, which is also r| r a , 
is of order 1. 

Example (Li), s, - jtj, which is 

V *!*!“ n^l Ja -f (1 - Jl )(1 - JTi) > 07 
— r*** -= j 3 "• > — n 

is of order 2 . (Proof that it is not order t is in g.11.J 
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hK ample Oil?, bet M be 9ft integer 0 < Jtf < Then the “counting 

ImiLtiflin” 

* M (X) -HX| - AT|, 

which reastgtireeB when X contains exactly M points, is of airier 2_ 

FhOCtf, Consider the lepreiseTitatLon 

-f(2jW - l}£i-+ (- 2)^P pr t jfj £ Af 1- ]. 

■Li iwtj 

Ecu any figured there will be | X| terms *,■ with vsLue l h and | X\ ■ {| X| - l)/2 
lerms a.-t, with value 1, Then the predicate is equal to 

* w (X)-nailf- 1) -|JfJ - \X\ -f|JC| - D+ 1 - Af»>(f| 

■.md the- duly (integer) value of \X\ for which this is true is \X\ - M- 
Note that the lineal form for the counting function does not contain 
H explicitly. Hence it works as well for an iuSnite sp H oe fl. Q.E.D, 
Example (iv). The functions f|X| £ Af "| and ^ Afl are of aider 
I because they are represented by f Z x . ^ A*1 and 
Example (vb W* can ohtain aft arbitrary function /l |A r |} of the area 
of a figure from the predicates used in fjy) above by wrttiti(( 

/(X) - flO) +- £ Iftk} -Hk- W-HXI >*-i. 

r>i 

L Jie order os a function can be determined by examining its representa¬ 
tion as a linear threshold with respect to sets of rr.askft To prove thifi we 
itrst show 

r l idr:oKi-:r.i (Positive normal Puhm theorem) , EWry ^ ts a linear (fcraaAoM 
junction with retptd to the set $f off masie. 

riltfOF. The well-known riisjunelivc-normal-form theorem for Boolean 
functions tells us that any Boolean function ^(JL ah ■ ■ m J tan be written 
in the form (DNF> 

MX} = Vh(X) 

ie J 

where 

fdX} = >ij y tj ."-y lh 

where for each i and j, y,. - *, or y (f = x r 
We can write this in linear form as; 

iHX)- rS*j> d“| 

ier 

because, for any X, at most Pric term of the ] W is mmtro. Hence, we can 
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replace Jo#ic;il "V" by arithmetic “ +Furthermore, sinon numerically 
■*. ” 1 — *i, each can be written in (he form. 

tiiX) - ‘ - - Xf J| - ■ - (1 - A, n ), 

suppoting that, the negative terms aie at the right. Multiplying this out, 
we obtain un oipression of the form 

MX) 

where Z,- j» of the form 

J| 1 ' ' ' *^**1 ’ ‘ ‘ i ^Ll ■ ’ -^n -1 C l'**!, r r r pt„ f. 

But such Zf are mash^ so that if; is a lineur Combination of masks, It 
follow* immediately that LS itself a linear combination of masks 

£ = ^tiiZ, 

where each a, is an integer and each Zj ^ mask.. Q.lii.i). 

Remark, The above Construction shows not only that any Boolean 
function is “lineal” in the set of masks in the '> ^ F > B~\ " sense, 

but- is also I incur in a stronger sense. It is interesting that 

this form is unique, and is therefore entitled to he called a "normal form." 
We till it a '‘positive norma] form.” To see the uniqueness, suppose that 

and consider the difference 

* = (^ o i Zi = J] $, Zj = fa) Zj. 

Now must he identksJly aero. To see this, consider ti&t any set 

.Y of one element Jr,, Then 

MX) = #(]*,)) - T| Ti |J, = Tij-i ■ 1 - U 

so Y|j,| = 0. Next, consider any two-element X = fjCjjtj); then 

" 'r|z JJrj ;|Xitj + T|lj|^+T| V |J^ 

“ *1^,1 ‘ 1 = 

eo all two-dement n^i'* arc zero. Similarly, by induction, all the y’s 
can be seen to vanish. 

The proof of the positive normal form theorem implies al£o the 

Theorem, f is of order A iff k if the fmailesl tm/nber for ii-hie.h there exist 
a jrer 4 1 of tnttxks satisfying 

|£(4>| k 

orvd 


* E Li*). 
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E* A Mi'Ll (vi). A "Boolean form” haft order no IviKlier than Ibe degree in 
LtiS disjunctive normal form. Thus 

so that the negations can he removed without raising order. This particular 
urder-3 form appeals later in a perceptron that recOgfikftu convex figures. 
Using this result we develop some more examples of the use of Hit concept 
of order. 

Theorem. If has order Q L and ha$ order Qi. theft f x ® arid 
4>i - two# order 0 , 4 - 0 g . 

Proof. The idea is to multiply together the positive mask representa¬ 
tions H — f].KSr*- - ft: j >0 | to get a positive form of order £ 0-, + 0j_ 
fUse -->■"■ f QJ — "<" for ® ,} This may not work in some Cflses whore 

£i — 0i or In such cases h it j& always possible to replace S L and 

(si by slightly different value*, algebraically independent of the coefficients 
of r, and 2 : , so that the predicates are unchanged but exact equality 
never hold*. 

App-froa-lion. 

Emhple (vhfo Since - rf|Jlf I £ jW ~l = H^l ^ M"11 > we 

conclude that $ u has order £ 2, the result of Example {jii>. 

^Mfslfon. What can be said about the orders ofP# lA^ s ^ and V <ti~\ 7 
The answer to this question may be surprising, in view of the nimpLe result 
of the previous theorem: it is shown in jjV that for any order n r there 
Exists a pair of predicates and both of order 1 for which A 

and (^V if^l have order > n. In fact suppose that R ^ A \JH\JC where 
A, H, and C fllre forge disjoint subset* of R. Then / 3 = |“|X D A l>|Jfncn 
and — P |Jf r"i -fi | > | X C| ~\ afoh have order 1 became they are 
represented by 

I £ i 4 - %> 0“| and HZ *J ~ S Jti > 0l 

a,ec ^ei *,^c 

but, as shown in }V, A V'j) acid {f 3 V have high orders. 

II, Croup theory of linear inequalities. In this section we consider linear 
threshold functions that are invariant under groups of permutations of 
the points of the base-space li. The purpose of this, realized finally in 
SV. is to establish A connection between the fieometry of R and the 
qUKCtfon of when a geometric predicate can be a linear threshold function. 

Aft an introduction to the methods Ltitruduccd in this section, we first 
consider a simple, almost trivial example. Suppose we wish to prove that 
the function X|Xj V Jc a i* not of order L To do so we might try to deduce 
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a wntradiction from the hy pothcsis that numbers a> & and J tan be found 
far which 

(1) - x L x E V 5]Fi = f axz + djij > § “I. 

We could proceed directly by writing- down the OGndttotte on a and £. 

T[ = 0 , i M - t? 0 > 

i, - Ji, x t = 0 =* a £ 0 , 

Xj = 0 , x t - 1 =£* p £ ft, 

X] = 1, * 2 = 1 a -i- a > 0, 

In this simple case it is easy enough to deduce the contradiction. 

Hul arguments of this sort are herd to generalise to more complex ritua- 
tiona involving many variables. On the other hand the following argument, 
though it may be considered more complicated in itself, loads to elegant 
Keneralizatlcms. First ohserve that the value of ¥■• ia invariant under 
permutation of and > s , that ia, 

^*n*sl = *(*s.if|h 

ThuB 


+ d*s> tf, 

all + dX; > 0; 

yields 

((a 4- 0)/2) xj + (( rt + j3>/3) Xj > $ 

by adding the inequalities. 

Similarly 

+ dx, 5 fl f 

CTXii — £ P 

yield e 

((« 4- x L 4- {{* + 0)/2) g 0, 

It follows that if we write y for fn + g)/2, then 

tH^l, X-i) - Pyx, -j- ■ y x 1 > 0 ~|; 

Le. r we can assume that the coefficients of x, and ^ in the linear repre¬ 
sentation of ^ are equal, it follows that 

lid-X) = Pf |X| > /"| or fy| X\ — f > 0"] 

(if we assume that the space X has only the two points Xj and ijj. 
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Now consider three values oF X, 

X a =A r |Jf n |=0, ?|JC|-*sO, 

A' , = | i;j | , |JC]| ™ I, 7|X| — ff>0, 

Jfs= f-Kj.Jta). |X 3 |-2 r t|JC|-«S0. 

Since A u and satis/y if.-, and Xi doea net. Che first-degree polynomial 
1 f| JCj — # Zn. | X\ would have to change direction twice, from positive to n^atiix 
SFH< back to positive OS | X| increases from 0 to 2. This is dearly impossible. 
Thu* we fearm MMnetAp^f sixNri i, by aierogin# ii over the permutations that 
leaoe ti InwirWiFft. The method is similar to that used Ln Haar measure 
theory. In fnct h for order I, it is the same method. 

The generalimtion of this procedure involves consideration of groups, of 
permutations on. the set H and functions £ invariant under these groups, 
of permutations. tn anticipation of application to gaoinc-trLcaJ problems, 
wa recall the mathematic! viewpoint from which every interesting geo¬ 
metrical property is an invariant of some natural transformation jjrmip. 
Let G be a group of permutations of Ri g£ l7 and XC R. and define 

x,- a\y\y = se. *EXi> 

= fMX,}, 

t = iv — i 3 g G Q f £ “ ^*}■ 

Thus we define- an equivalence relation of Vs with, respect to a croup G, 

The GnotU' Invariance Theorem. Let 
ti) G be « finite group of pErmytaijans of R; 

fill 'two sec of p red ieatdS tur R closed under G, j.e., i £ to, g (£ G =±> g> f £- 
Siu) £ in and Cnuariant under G. 

Then there exists a linear representation of 

* - TL $.4 > ol 

for which iAc coefficients fi t depend only on the t'Leqniwitenve dost of a-, j.e., 

* - c V ^ jfl, — - 

fttOQF. Divide +■ into equivalence classes by the relation = £•: 

♦ * 4|U U %i. 

Now let £ = rX*E+ CT #^^) ^ ®1 he any linntr representation of £■ and 
Chuosc X such that j,c H > G- 

Since tf-(X) — it follows that for each 
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X (^*1 = X ) > 0, 

+e* *t* 

the sum of positive quantities Ls positive we can Bum aJJ such equations: 

L Z > 0. 

*E(i+e* 

Since -t-- U f. i r the expression on the Juft can he written: 

* h 

2 2 si ^ si 

jfeC-^-i*e*i <-i i£<? *e*v 

Hence, 

Z Z (Z > c- 

t-i *eti\(£tj / 

Now observe that the set 


*ie= \ *$\*e +j \ - 1 #|* e *j| ■= *i 

because any £ just permutes members of an equivalence claw. Then also. 

■fr. “ *tg~\ 

Hence frw any £ 

Z SI ar+e J = £* # *. 

+*=♦■ ft*,! - * *e*n 

So 


z 


si * f <&* 

*C*I 


SI £ %-if — 

I 





Sineo mu rune over 0, "covers" # Sw , then Sl iE o^-i has the Mine 
value for a]] equivalent *’*. he., if *£*„ Zreo^-i depends only on I. 
Therefore we Ctn denote Zrfc£i*«-i by fl ,- obtaining; 

* 

Z Z ftfix) >o 

i-j ft*. 


or 


Z^e(#k#C^f} > 0, 

where t'(*) denotes “‘the equivalence claw containing p, 1 ’ 

h similar sr-jtumeni shows that if 2 **♦<*> <£h then 5 > to ,*{X> 

< ft Thus if> " HZ fs * ♦ > Q"1 “ rZ^£ui & > 01- Wfc shall moat often 
use this. theorem in the following form: 

OoBOLLAlrv 1. Any funr.tuin at. of order- it has a fcfmr rFprejsejaiiCirifi. 

* - rz«*+><n 
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tehfrt t is the. set of masks of degrees -S, k cifjtil a t — a t wherwer $($■) cori he 
transformed into S{$') by an element of G. 

PROOF. The corollary follows immediately from the theorem and the 

observation that, for ^*=13^ if And only if A = H M for some g t (T 

Cohijluiht 2. Lei * =• *i U " ■ ■ U *.ti be the decomposition of fl> into 
equivalence classes by the relation - e _ Then if f is in Lfi) and r is closed 
■under G r if ran Ae written in the form 

*-rz>.-NiUo>0i 

ichere N,-{X) - | #,■; j j| ( i.e., Ni{X) is the number of « a of the 

i-th type, equivalent under the group, that "fit" the argument X. 

Pm OOF. £ can. be represented as 

<i> = |~Z ft* <t* > on 

“ r zi 

■ *=*i 

- rz=' n * > 9 i - ><n. 

v +et* 1 

('UE3U.LLA.lt.V 3. (ThS TaiVIAUTTf U^ iNVAHIAhT PsLKIliCAlBfl V¥ QUIkt 1). 
Let G At 1 any transitive group of permutations on R {fftfctotdiw means: for 
every pair p,q^z R there is a g£G such that pg «■ q). Then the only first-order 
predicates invariant under G are of the forms’. 

MX) - riX| > 

or 

MX) = P|X| < for some m. 

Phoof. Since the group is transitive all the one-point predicates £ if | 
are equivalent. Thus we can assume that 

- rirf=wif!> s ■ (or with some other inequalit y ftifpd 

i.e., the coefficient a is independent of p. Bui ZpeJt^jpl > # can be 
transformed into Zr€A*iM > fl/o (for a > O' for a £ 0 a similar argument 
proves the Corresponding assertion), But ZpeA#sjj| " i^l- Thun order-1 
invariant predicates can do nothin# more than define a count cm the 
cardinality or "area” of figures. In fact, an ordoH prediestt is a measure, 
and Ihe order-1 invariant- predicate is the Haar measure. 

III. Applications of the yrmup-ltivarinnee theorem. 

The Parity Function. In this section we develop in Home detail the analysis 
of the particular predicate >£hih defined by 

=■ p| Jif | is odd - ]. 
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Our irttettal in +TJ& ia threefold: it its interesting in itself; it will be used 
ibr the analysis of ether more ampoftarit functions; and, especially, it 
■UuBtxatH «ur mathematical methods and the kind of question they Enable 
us to discuss. 


THKMM, is of Order |ff|. 

That is, to compute requires at least one predicate whose support 
covers the wtefe space ftl 

Pkoof. Ut G be the group of all permutations of R, Clearly ^. PAR is 
invariant under G. 

Now suppose that where the are masks with 

I -5(ii) I £ ft and theu, depend only on the equivalence classes defined by — s 
Since musks with the same support are identical, 

tf, = o | - | &'{$,) |. 

Thus 

* , , 

■h-MA - fE H H >&] 

i-o \ +e*j: / 

Where «fr. is the set of masks, whose supports contain esaetly j dements, 
He now calculate for an arbitrary subset X of 

CiW = I *<XI 

♦etj- 

Since is i if S(^j C X and G otherwise, C,.fX) is the number of sub¬ 
sets of X with j elements, i.e., 



which is a polynomial of degree j in |A|. 

It follows that 

K 

^ *j C,[X} 

J-u 

is a polynomial of decree X in |Xj, guy P(|X| ) r 
Now consider a sequence 

A-XnCX.C— = R 

uf !.JS| +1 nested subsets of ft, and the sequence of values 

Pf|Jfa|)-O t =1. =Q,... t Pi\X m \). 

This implies that changes direction f J?| times as| X| increases from 

0 t* | J?|. But since Pis a polynomial of degree ft', it follows that K - | H\ 
Q.E.D. 
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From this life obtain the 

I’heorem. if ^j>ar-E i(+.) and if * canto ins miy musks, Item <l contains 
:ill the mmks, 

Proof. Suppose, if po&ftiWi\ that ft™ £*.(*), that. + contains only 
masks, and the maak who** support is A does, tint belong to +, 

l^t ft'A^I “ I > t 1 ”], f^Gne, for any <fi, - MX H A ). 

Clearly the parity function fm subsets of A, is of order \A\ by the 
previous theorem, 

Now consider for #e *. If 5U) C A, dearly <t>* - & If Si*) ie not 
a subset of A r ** is identically zero since 

si*) $ a $w c x n a nx n a) = o =* * A vo - o. 

It follows that either S{* A ) is a proper subset of >1 or m>* Js identically 
zero, Lot ‘i' 1 be the set of masks in 4 whose supports are subsets of A. 
'I hen *e* ^ > 0 _ l■ But- for all ^ |S{#)J < | A|. It would 

follow that the order of is less than |A| r which is u contradiction. 
Thus tlse hypotheses are impossible and the theorem follows. Q.E,D r 

Corollary 1. // £ Jt-(■*■>, then 4 must contain al least ojjf * for 

U'A EC A 

l-S(*)| - \R\. 

The following 1 theoiecu, also immediate from the above is of interest to 
students of threshold logic; 

CoAOLUinr 2. Let 4 be the se( of all for proper subsets A of R. Then 

I he follow log theorem gives a hint that certain functions that might 
be recognizable, in principle, by a very large perceplron, might not. actually 
be realizable in practice because of huge coefficients. 

Coefficients of the Parity Function. Suppose that wc liave a > 0"| 

that, recognizes Parity (| X | I with, ansisks. Let us suppose that the recognition 
is neiEofile, e-,g„ that for odd parity, and , o,- < fi for even 

parity, if we apply the full permutation group, we obtain the same reliable 
discrimination with a set of "average coefficients" &; all equal for ft 5 -of 
the same order. Then we obtain the inequalities 

*2 + 2ii ] < U | or £ ^ J Oj > 2„ if n is odd,, 

Oj 4 dorj + 'ta t > 2 !> < 0, If A ie even. 
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Subtracting suoutedve inequalities, define 

—«+?[ CT 1 ) -(T) ]— -4 G-J «■ 

-?(:)- 

so that for 4 i 11 Jt, 

oj f(-1J-0.-2] >0. 

Using theae inequalities, we will obtain a bound cm the coefficients 
w * will sum the inequalities with certain positive weights: choose a ny 
M > 0, and consider 


Then 


Z (f) It- 


The left-hand aide is 


ss<-*>■-(;) (T) -S&-MO (i) 

-£““*(") i-a*a-!)■“-* 


= **,,{- 1) W 
a? we have the 
Ti 11 on k.n . Far corft M. 


{- l)^ Ntl > 3* + . 
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These values bold for the average, su if the coefficients of each type are 
not equal, some must he even larger! This shows that it is impractical 
to use mask-like *"& to recognize parity-like; functions; tVlh if one could 
afford the huge number of p's, one would have also to cope with huge 
iUnj(££ of their coefficients!! 

Kehakll. 'This ti»E a practically .fatal effect on the corresponding learning 
machines. At leant 2 1 * 1 instances nf jn-.i no maximal pattern is required 
to “learn” the largest coefficient: actually the situation is far worse because 
of the unfavorable interactions with Lower order coefficients. It follows, 
moreover that the information capacity necessary to Store the set | <r -j 
of coefficient* in greater than that, needed to store the entire set of patterns 
recognised by ^ PU! —that is. the even subsets of R . For, any uniform 
representation of the ct.'e must allow j i?| bits for each, and since there 
are 2 ,fi| coefficients the total number of bits requited is |i^| ■ S |fil - On the 
other hand there are E 1 * " 3 even subsets of fi, each representable by an 
mi ■hit sequence, so that 1?| - 2 ifi| 1 bits wouLd aufface to represent the 
subsets. 

It shouLd aleo be noted that- ^jvhh is not very exceptional in this regard 
because the positive normal form theorem tells us that all possible £ llfl 
Boolean functions can be so uncoiled as linear threshold functions in the 
set of all masks. Then, on the average, specification of the coe-fficienlE of 
each rec [Hires 2 JJ| hits. 

Another predicate of great interest is associated with the geometric 
property of “connectedness:I Is. uppLicjitinai find interpretation is deferred 
to §V; the basic theorem is proved now. 

The "One-in-a-box" Theorem . 

Tubur™. I Jit A Lj ■ be dispirit subsets of it and define the predicate 

HX) - r(Vi}{|xriA ; | >on 

ij. r there is aZ tecs! one point of X in each A.. Then if for oil i, | A, — 4m s , 
the order of ^ is £ m. 

Odhuart if X ■= A, |j A e |J - IJ A„ p the order of -p is at least the 
order of (lflj/ 4 )^ 

Ffeaar. For each f = !,■■■,« let Gi be the group of permutations of 
R which permutes the elements of A , but do not affect the elements of the 
complement of A,. Let G be the group generated by nil elements of the 
Gjr Clearly ^ is invariant with respect to G, Let $ he the aet of masks of 
degree K or lees. To determine the equivalence class of any <p <E consider 
the ordered set of occupancy number* 

!|.5<0nA ( | l 
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ThetJ *i = 6 ** if, fur each i, |$(p L J nA,| = |$(*J nA,| - Let +,, +, h 
ba the equivalence classes. 

New eunsider an arbitrary aec X end an equivalence class r, Wu wish 

t« calculate the number jV.fXl of members of *, satisfied by X, i,e., 

iV.(A r ) = j j *| * fz *. A #{+) C X [I - 

A simple combinatorial argument she™* that 

\ / i ^ i^sf \ / |JfriA. v i ^ 

\|sc*)nAi 1 |/ "' \\Sfo) nA^i ) 

where 

_y(y- l) (y = n + l) 

\nj ~ n! 

And 4> k an arbitrary member of ^ Since the numbers | £<p| f| Aj| depend 
oniy on the classes $ } and add up to not more than K, it follows that JV,( X} 
can he written as a polynomial of degree X tnr Jess in the numbers r, — | X A,| 

JVjWO-p, p ^j. 

Kow feW - | > 0 , be a representation of £ as a linear threshold 

function in the set, of masts of degree Less than or equal to K, By the 
argument which m havE already used several tuncE we can assume that 
“+ depends only on the equivalence class of » and writE- 

£<**#<*) = £> L #{X) - 'Z^N.iX) 

1 -J *-£*; j-l 

H 

-Hfij "‘■rJCj 

/-l 

which, as a sum of polynomials of degree at moat A', is Itself such a poly, 
normal Thus we can conclude that there esi&te a polynomial of degree 
at most K, 

Q(*n **•,*■) 

with the property that 

•tiX} — | f?{3f n - - xj >0 with x, -= | X fi A,| ~\ 
d.6., that for all i, 0 ^ Jtj £ Am' 

^(*l, ■ j 'i >0 ^ £ Vi) > 0 ). 

Iu ^(Sj, ■ ,jf„> make the formal substitution, 

(t- £21- 1» # , 
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Then ^(j Li becomes a polynomial of degree at most 2 K in i, Now 

]ct if take on the values I ■= 0 P l p “■ - r 2m. Ky property {? must he positive 
for even t end negative or £cru for odd t, By counting the number of changes 
of sJrjj it is clear that 2K j; Sat i,c., K S' m. This completes the proof. 

JV. Tbe and/or theorem. We have already remarked that if K = A\JB 
U C the predicate 


(JO - r I X O A J > \ X O C| | is of order 1, 
and stated without proof that 

■■p{Xi . f |*nA| >ixnc| A\xns\ >|xncn 

is not- of bounded order as |fi; becomes large. We shall now prove this 
assertion. We can assume without any in**, of generality that |A = |£ij 
= |CJ e.nd our forma) statement is that if ft(X) is the predicate of the 
stated form for | * 3ft, then the order of #*-» = as k . The proof is 

dmitar to that used for the purity theorem. We shall assume that the order 
Of (■#,> is boundsd hy JV for a]] ft ana derive the contradiction bv showing 
that the associated polynomials would have to satisfy inconsistent condii- 
ItonE. The first. step is to set up the associated polynomials for a fixed k. 
Wc do this by choosing the group which permutes within the sets A, £. 
f. The equivalence classes of Jnikks are then characterised by three numbers, 
Le. r |A n $ Itf)1, |Bn5(*i| and | Cn-Sfrfi) |. The number N+{X'} of masks 
in this equivalence class saLisfied by a given set X is 


N,m = ( |j4n *' ) x { 1 


/ icruci \ 
\\cr\Ste\/ 


It £ N t-his is dearly a polynomial of degree St most N in the 

three numbers 


*-|AnJf|, y^[Bnx\ r *=|cnA T j. 

The group invariance theorem says that if 

f» " I Z T#+> 0 "| 

*e* 

when 4- lh the set of masks with |5(>)| S j\', then 

hi*) - 

where f runs over the set of equivilgnce classes: of <t>. But Z*J MAX) Is 
Si polynomial of degree at most JV in >' and /. CaLI it /**(*, .kt). 

Now, by definition, for possible values of x, y, a (i.e., normegativv integers 
JA ■?) > d it and only if i> r and >■>?, We ahaEL show, through 
a series of lemmas, that this cannot be true for all k. The technical details 
of these lemmas are not essentia) for the subsequent sen;lions. 
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Lemma L Let l\it r y>z) be an in/rruA? seflueiu* of polynomials of fizttd 
dffZW jj, tolth Me property Lhut for nfi pcsi'lrue integers x, y, z ten- Men h r 

... *> t and y> z =& P t {x>y t t) g 0 h 

(A) 

JS t or y * t P*U p _y F z) < Q_ 

Then chore e* isU g nonzero polynomial Pi.x,y.zf of the %$ina degree h with 
the property that the implications (A) hold for all positive integral values? 
ot *. >', 2 . i’h* follows from the following eontpEielness argument: Write 

I\Uiy.z\ ^ V CV.-ntttayv*) 
i-j 

where j 'n [ {x, T y l z} is :.m enumeration of the monoisifels in variables jc. y 
and z. We ran assume 1 since the hypotheses remain true jf P k jb 

divided by Now the hounded sequence C L | P Cj i( , - ”, Cgi, ■ ■ ■ 

must Contain an infinite convergent subeequeiK* $■, of the integers for which 

i Cm I ^ €: Si | converges to a limit, say C 3 . 

Kow consider l Cjti| ft-6 [. There must be an infinite subsequence of 
Si ¥ say Sj, on which this converge |y u limit, say C 2 . Continuing in this 
way we find A subsequence S — S', of the integers And a sol of numbers 
C|^-C, such that ftVdl^ESj converges to C ( for all r But lbcn h 
for kfES, PAx r y,z) converge to tb* polynomial 

r 

P{x.y. z) = X- C n*; ( x h y , f> for all i, y, z. 

■•"I 

To Aw that P{x,y,z) has the required properties, choose any positive 
integers iy, y { , iy, For values of k smaller than the largest of these numbers, 
nothing can be said about JP*Uy,y*,z*}- But for all sufficiently large it, 
P^- Jt y Ql ?n) must he nixnnegttive jf i, > ^ and y 0 > z D taftd fttna positive 
il x 0 < z 0 or y 4 < i u ). It follows immediately that P{x D , vy, s*} is nonnegative 
(nanpoattlv*) under the same conditions. To see that P is? nonzero note 
that^ej 1 = l r 

Lemma 2. If a polynomial fio. d) .tutufit* Me filming conditions for all 
integral aai'oes of o and Men it is identically ze no: 

(B) a > 0 and ft >0 => fio.ff) £ 0, 

(Cl w £ £■ or & S t> -± f[a,&) i 0. 

PROOP. Suppose that a polynomial of degree N t fta^S}, satisfies the 
oo ltd it Lons (B) and (C} and is not identically zero. WLchiuti lows of generality 
we can suppose that 

- a N gU}) -+■ r(a,d} 

where girl) is not identically zero and rfn H d) has degree Jess than S in *, 
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For any i3 for which £03) ^0, there is an eo, > 0 such that 

> |jWfl}j. 

Thns f{&<,,$ lw& the fiame sign as i.e., m gifi) emce w positive. 

It follows from (B) and fC) that 

A > to =>gip) g o, 

A < 0 Jftd} £ 0 . 

(The conditions fD) hold for till jd; if y 0 by preceding argument; 
iftf(iJ) -0, tautolojouelyj We now derived Contradiction by considering 
separately two eases: 

fa) fV cimn, Since is not identically iem, there La some flj > 0 for 
which eW * 5- By (D), Thus a N g(&j) > 0 so that for |<r| 

sufficiently large 

+r(ft»AJ > 0 

i.e- / (cr, fl(,> > 0. iBut we are free to choose a negative 1 value of a p i.e,, we 
can find er+, ;S,\ such that 

eta < 0 and f > 0 

which contradicts (C)„ 

(b) N odd. Choose fti <G for which tflflj) ^ 0; then gifftf} < 0, by (D). 
Cbowe negative a 0 as before. Then aajffCftJ > 0 and > G h again 

contradicting (Cl- Q.ELD. 

Ijjkha 3- iVo nonzero polynomial P{x,y,z) can satisfy lAe /oifowi^g 
conditions for a£i posiliw itiie^nai ittfues of x. y, z. 

x>Z and y > i P{x,y,z} £ O, 

nr y£z-*PtY,y,3) | 0 . 

Phoof, Suppose that P(j£,y,.?Ji bss these properties. Define- £?)**„ 0. a) 
-r F{z + a r z + j} T j). Let M be the highest power of z 3(5 y ao that 

where Ii is of degree Less than A# in z. 

Now choose any and {i a for which f(a t ,fafr ?f£). For autheienti) 1 Large 
z, say a? 

(a j ^ + «.] > 0 and *o + ft] > 0, 

( W | Zg /(.wDido) | ? | ■Rf.Oftiiflni^u.) | ■ 

It follows that 

/{&*.&[) 0 ■; 0, 

■F'fi-ii + + ,i3[.. Zn) ^ G, 
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Thus 

*a> 0 and fl„>0 —9- *» + 07 , > 3^ h and f fl + ft > Zg 
= J^ P{Z(f + Cry,- A] + ft, £t) ^ 0 
-* £, 0 

and similarly ^<0 or ft <0=S*/(« fl ,ft) i 0, But this is true for all 
ft* ft. Thus by L-be previous lemma, fi*,?) = 0. It follows, that Pix r y r $ 
is of degree aero in which is only possible Lf it is identically sero. Q.K.IX 

This concludes the proof of the AND-OR theorem, It- La dear that the 
reason the tneorem is Liue has to do with the algebraic geometry of the 
“occupancy" polynomials. If it were not for the constraints concerning 
integer values of tho variables, the theorem would be an immediate con¬ 
sequence of Beiout’s theorem. 

V, The J "urdcr-limilcd” perccjUrtus. 

The Order of Some Ghftmein'osi Predicates. Now we consider the problem 
of computing the order of a number of interesting; geometrical predicates. 
Aa a first step, we have to provide the underlying space R with the topologi¬ 
cal and metric properties necessary for defining geometrical figures;, this 
wtis not nftCftHHUy in the case of predlcarea like Parity and Others related 
to counting, for these arc not really geometric in character. 

The simplest procedure that is rigorous enough yet not too mathe¬ 
matically lussy seems, to be to divide the Kudideon plane, E t t into squares- 
as an infinite chess board. The set R is then taken ns £fo: .w£ i>/ ^tutPW, A 
figure X of E* is (hen- identified with that set of elements of R —i.e., that 
collection of squares—that contain at least one point of X . Thus to any 
subset X of E 2 corresponds the subset X of R defined by 

* = (A€A|«nx^Aj. 

Now, although X and X are logically distinct no serious confusion can 
arise if we identify them, and we shall do so from now on. Thue we refer 
to Certain subsets of ft as "circles, 1 ' “triangles," etc., meaning that they 
can he obtained from real circles and triangles by the map X^X. Of 
course, this means that near the “limits of resolution" one begins to obtain 
appEirenl errors of classification because of the finite "mesh” of R. Thtm 
a Anigll circle 



will not look very round. 
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When ill is necessary to distinguish between i 7 ' and F we will say that 
two figure* X, X' rtf fc’ 1 ere- in the- sumo ff-tolerance class if 3( = X'. In. 
this we follow the general mathematical approach proposed by E. C. Zeeman 
for tPMtiJlg (IiLe kind of problem. 'To avoid in««sct!»r.i».! i|m>iions of how 
the group-invariance theorem applies to infinite groups, assume below 
when necessary that R has the toroidal topology. 

We begin by Listing some geometric predicates of rather small order, 
fa) £ = I. When we say “geometric property" we mean something that 
ieai Least invariant under translation, usually also invariant under rotation, 
and often invariant under dilatation. The first two invariances combine 
to define the “congruence" group of transformation? and all three the 
"similarity” group. For — l h just the translation group suffices for the 
Group Invariance Theorem to tell us that ail coefficients are equal, hence 
the only patterns that can he of order t are those defined by a single cut 
in the cardinality or area of the set: 

*!. — riA'i> a ~i pr t ~r\x\<A-\. 

Note: If translation invariance is nul required. then order-1 can compute 
other properties, Le. r concerning no/taercrs about particular points Or ases. 
However theta ate not "geometric," 

[b] k - 2. For k — 2 things are more complicated. As shown in $1 it 
is possible to make a double cut in the area of the set, hence we can do the 
counting trick, and rncogtLrae 11 k.kw figures whose areas are 

P = _ Ai < | f-’| < A^\, 

tin fact, in general we can always find a function of order k that recognises 
the sets satistisd by arty k inequalities concerning their cardinality.) Now 
coivsidot only the group of translations and masks of order 2. Then two 
masks XjJta and xjxj are equivalent if and only if the difference tetipr? 

i L — jc £ and Jcl — ij 


are equal- Then, with respect to rhe unmsJ^t.iori group, a figure is completely 
characterised (up to k — 2) by He "‘difference-vector spectrum,” defined 
as the sequence of the numbers of pairs of points separated by each possible 
directed distance. The two figures:: 
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have the same difference-vector Eppctro,. hence no order -2 predicate can 
make a classification which is both translation invariant and separates 
these two figures. Similarly, 



have different difference"vector spectra. 

If wo add the tefiuiiement of invaritwWt under rotation, the last pair 
idjnjvo becomes indistingUiiEhBbkc, for the spectra now relevant classify 
together all differences of the same length,, whatever their orientalion.' 
An interesting puir of figure* rotational)}' distinct, hot still indistinguishable, 
for k -■ 2, is tl« pair 



which have the same direction -independent distance-between point-pair 
statistics- There is an interesting theoretical direction here, hut we will 
not stop to look into it.Many interesting proposals for pattern recognition 
machine aro related to the theory of tliese geometric spectra. The classic 
paper of Bledsoe anti Browning |1| Is related to this, as is the work on 
L 'integral geometry" of Nnvikaff ,<]. 

te) If — J. As 4 increases. the class of rcisli mbit 1 discriminationE grows. 


J NnteLha: we did .nof allow iriflrrlitjnt. jet lli4W' flshwtu-inialLy oripusUe li|;u>reh ara ns«' 
cmriKdl Urw should. Ik cautwui nt-imC ue+iii£ ■ipncuition'" bore. Th.*- rhwjry of mtaLionaj 
■iii'A riAn$e f(Hpj-r™ CArtfbl AllAiitioA to the tfftcL uf llu- duscroL-H ffttinA. AppnutmatKin, 
ni.i cun pruciiindhl v tw mndc cunsuccnt hv apcjicatiun Of Eefiosm's rucLbtidj; tui Lhe (hlsti 
linn 'cruup,'' Lhtrr are kcwui ttillimiliw 
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and our detailed understanding wanes. It is interesting to discover that 
the predictte 

e(X) = |~A is a single, solid, convert £igure""| 
is of order S3, as notnd in the Introduction, because 

£comve* 1X) ■ TZ | o € X and b£X and. midpoint (□,£) {£ A - ] < l~| 

is of order 3. Presumably this predict!* cannot be realized with order 2. 
]| Ls not difficult to show that the set of solid hMtsngle* (with axis parade) 
to the mesh of -ffj tan bo recognised by a predicate of order 3. This is true 
also for the set of hollow rectangles (with borders one square thick), It 
is much more difficult to show, but true, that the set of hollow squares ban 
order three’ Intuitively one might suppose that at least order -1 Lr lequirtd 
to Insure equality of side length &. 

Another example of w predicate that can be realized with k — 3, for 
any n, La 

the points of X are collinear, and broken into 
riot more than n segments - ! - 

{d) h - 4, Using the fact that any three points determine a circle, we 
can make a perception with mask* of order k — 4 for the following predicates: 

•f- iX 'l — | X is the perimeter of a coiiLpleteci relftl ■ 

Proof. 4 Define, fot all concydac quadruples of points in fl; a, &. c T d 
{X) - fa £ X and b £ X and c£ X and d £ X "| 
and then realise 4> as 

# = r Z < 1~| ■ 

-r.fe.-r.d 

Many other curious and interesting predicates can he shown by similar 
argument* to have small erdens- Une should be careful net to conclude 
that this means that there are practical consequences of this, unless one 
is prepared to face the fact that- 

(a) largo numbere of ^'s are required, of the order of ‘ for the examples 
given above. 

in) the threshold conditions are sharp, so that engineering considerations 
may cause difficullirs in realising the linear summation, especially if there 
is any problem of noise. Even with simple square-root mine, for k “ 3 
or larger, the noise grows faster thnii the retinal seze. 

4 An flitcEimlivc rneLhiid is lo uitc-irate Lbt curvature tif i;ne elements. Thin leans In mler- 
ratbiE qriMtHirts. about the jpndalffn el global. imnctKirw thet es-n b* apjjr-o-mrnBte'd hy sum- 
nwLbtm of Incu elements wilh a eweii |:rsci«i*>n Curvature requires order 1, in a sense. The 
i: re:l::-.’ I >■ defines! here edir>lr« a IV*' uimiteiwiUni: esceplione. 
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(e) a very $i« K ht change in the pattern-definition destroys the mw . 
mzabUity, 

Furthermore-, in most eases there will be more efficient machines, for the 
* me “ m 0Ltnl hardware, Id realize those rather simply-defined patterns 
Low-order recognition has often the character o t a "tridt,” and one cannot 
Roneraiiise I'mcly. The AND-OR order theorem te-JJs us that some simple 
relations between simple properties of figures can be prohibitively hard 
to recognize, 

vj - rouMrtieitj: A geometric property with unbounded order. We define 
WBAeciedwss as follows; 

Two point* of ft are adjacent if they are scares I in tlic map F->F] 
wati-, a common edge. A figure is connected if, given any two points P Lr P, 
of the figure, we can find a path through adjacent squares from P, to P s '. 

Theorem. The predicate 


fEAj = | A m connected ] 
hot arbiirarify large orders or | A 1 1 grows j o 3/40. 

^ PaotiF. Suppose that (X) could have order- < m . Consider an stray of 
(2nt-f 1) X4m' adjacent squares of R arranged in 2m + 1 rows of -1 m 1 
squares each. Let f,’ u be the set of points shaded in the diagram below 



L.e., the array points whose tow indices arc odd r and Let, P L bo the remaining 
squares of the array. Let J* be the family of figures obtained from the 
figure Cc by adding subsets of G v It is clear that if P£ -** it is of the form 
G*y F h where F l CG l - Now FiriJJ be connected if and only if its F, con¬ 
tains at least one square from each even row- that is, if the set F : satislies the 
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“one-in-a-box” condition (see end of Jj-3). The theorem then follows from 
the One- in- a-B i'js. Theorem. 

I o see the details uJ how the One-in-a-Boi Theorem is appLLed, if it is not 
already deal, consider the figures of family 5* as a subset of all possible 
fiflUITte on Fi Clearly, if we had an o:der-£ predicate that could recognize 
connectivity on we could have one that worked on 3*- r namely the same 
predicate with constant zero inpnta to all variables not in the small array. 
And since all points of the odd rows have always value I for 6 gyres in J#\ 
this m. turn means that -wo could have an nrder-A predicate to decide- the 
one*-in a-bo* property on set Ci; namely the same predicate further- n>- 
strictfd to having constant one inputs to the points Ln O v . Thus each Boolean 
function of the original predicate is replaced by the function obtained by 
fixiriH stimt of its variables to 3eirt> and one: this operation can never Increase 
the order- of a function, But since this hist predicate cannot exist, neither 
Can the first. 

An Emmpit. Consider the special case for k — 2, and the equivalent 
one-rn-a-boji problem for a fj'|-&pace of the form 


p 


fl 





?-:>• 

X-' 


% 






d 


b 

b 










En which m - 3 and there are juat 4 squares in each row. Now consider 
■i - o! degree 2, we will show that it cannot chsraeteriice the connectedness 
of pictures, of this kind, Suppose that 4 - f !>,<*, > #1 and consider the 
equivalent form, symmetrized under the full group of permutations thac 
interchange the rows dtwf permute within rows,* Then then are just three 
equivalence-classes of masks of degree £2, namely; 

single points: ■= i ip 

point-pairs: = ±. X) ( Xj and Xf En gam* row), 

point-pairs: {n c and j, in different rows); 


Nutelhii lbi*» Hithtianii mm iissd in proving tbti qcrM-ral thwrcn. 
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hence any order-2 predicate must have the iorm 

(1) 4 = ViN '{X )-|- * U AT 1 L ( X) -j- * uiV “{X} > « 

where jV, N u , and N"’ are the numheie of point sets of the reaper! ive: types 
in the figure X. 

Now consider the two figures: 




= A ri - 0, jV if -9; 

hence the form (I) has the same value fur both figures. But X, is connected 
while X-j. is noli Note that here m - 3 so that we obtain a contradiction 
with I>1 j| = d, white tho Kvncial proof required lAj - 4 m i ^3S. it is 
known also that if k w 6, we can get a similar result with j j4, | - 36, 

The ease of £ = 2, !» — &, |j 4,] —3 is of order 2, since one can in fact 
express the connectivity predicate for that gpjtce u¥ 

4 - f A ,l {Jf) + A'^IX) - 2jV 11 IXJ > 4"|. 

ttit-icisfl Connectivity. It should be observed that the proof of the previous 
theorem applies only lo a property of connectivity in its classical sen&e 
hut to the stronger predicate defined by: 

A figure X ia "cutwiae disconnected” if there ie a straight hnt L tuch that; 

F -does not intersect L and daw wt lit tnti-tvly fo one side of 

The general connectivity definition would have “curve” for L instead of 
“straight tine/" and ube would expect that this would require a higher 
order for its realization. 
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Hfintifm.'t tiiilb.tfiti /-VnjepPWW. The study of the -order of predicates is 
often facilitated by the reduction of a given girttiittlt to another simpler 
one. Although we do not have a satisfactory theory of any class of reduc¬ 
tion. 1 ^ or even u dour enough insight into the nature of the relations which 
might play a role analogous to " homomorphism“quotient” and so on 
in more developed areas of mathematics, the following examples ore useful 
in particular applications and Indicate an interesting area for future research. 

|al Let us say that a perception system, P r is defined by the basic set 
if and a set ^ of predicates on subsets of it. A second perceptrctl System, 
P J , is u subpcrccptron system of P if the basic eet R' La a subset of R and 
if Its set of predicates +' is that obtained by rclativising the members of 
+ to fl' h L.e,, all predicates satisfy 

X G R ' ^ P’ (X) — 0 (X) for some + 

and all predicates ft satisfying this condition ate in <b\ Clearly the order 
of any predicate yf the form 4> f for F* is at most that of <fi for P. 

<b) Isomorphism must he Riven the following natural sense: Let P be 
defined by R and and P' by if' and 'then an isomorphism, /, is an 
isomorphic map /: R — R’ of the sets R with the property that for each 
#£=* there us exactly one + J £ + satisfying +(#)-#'(/(*)) {where fix) 
- (pe fl J | 3gGR[/(a> -pj). 

(c) P' is obtained from p by a cpUapaing operation /, if / is a map from 
points of if' to disjoint sets of R, i.e., 

PG if -*fip)CR, 
p ft f(p)nf{a\ = k. 

A predicate f J on R' is obtained from a predicate ^ on if bv the collapsing 
map / if * r (X') -*{/{*% ibr 

Theorem ((.'ollo esln u TuiOasii). // f is a collapsing amp from if to 
ft' and f is obtained from, a predicate f fry f. then the o^rfer of is pot greater 
itt.'i that of f. 

Proof. Let ^ — T > U~| where # ia the set of masks of degree 

less than k on R. Now for any X' C.R', 

*W = tifVC'}) 

# 

We next observe that (11 remains true if 4- is replaced by the set $ of 
masks f for which CW). for if 


#{/(*'))«[> for all X C S'. 


N ow for ^ ^ we have 
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*WCU| ApHpG^I, 

in fact 

c U ) n*(*J ^ a|. 

Thus, 

X r ~2\ P\fip) Hi!#) ^ .\ | 

-* ttX>) DU|/(p)| * A j 3 ,(+) 

i-e-r Jf' D (pI /(p) n fl W A [ -*> /( XI *(f{X' )). On the other 

hind. If i.fi., f{X f } it follows that 

/ip) r>flW yt A X' 

aince/(p)nf(fl) = A forp* fl , Thus *</<*'}} - \~X f C [pi f{p) C\* M * Aj~|.■ 

Iri oilier words is a mtsk on ft' 1 with support 

\ P\fiP> O »l>) H A |. 

Hut smce thfi «t* wf form/(pi are disjoint, for different p, it follows that 
H Pi ftp} naM ** Ail i |a(«)| < k. 

Goins hack to Equation {1) we see, then, that 4 ' ls represented as a Linear 
function of masks of -degree lcse than k. Q.F,.D. 

Huffman-5 Conitnu-.tajn for Vi’r shall illustnle the application of 
the preceding concept by giving an alternative proof that 4^ has no finite 
order, based on a construction suggested to us hy D. Huffman. 

The intuitive idea is to construct a switching network which will be 
connected if an even aura her of its ft switches are in the '‘on 1 ’ position, 
Thus the connwiedneaa p rob] Cm is reduced to the parity problem. The 
network is shown in the diagram for ft “ 3. 



Z 

— 

A 1 

*1 


*1 



-H 

—- 

** 

—— X s - 

' -t.1 

-R--- 


I in 1 interpretation of the symbols jr, and x^ ls as follows: when if,- is m the 
jOn" position contact is made whenever *, appears, and broken whenever 
X. appear*; when *, is in the “off 1 " position contact is made where I, appears 
and broken where Si appears. It us easy to see that the whole net is cmi- 
nectcd in the efefttical and topological sense if the number of switches 
in the “on' position is G or 2 . The generalization to ft i* obviousi 
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W List the terms in th£ classical normal form for considered as 
a point fonctiio!i h which in the ca&C n m-n. can be written:: 

^fW j I’” *J * ‘ ^ VjCjiiia x t V 4£]J£s ■ ■ ■ 

i.bi Trauslale this BooLean expression into a switching net by interpreting 
conjunction as series coupling and disjunction as parallel coupling. 

(c) Con&lruct a percept ton which “Jocks at" the position of the twitches. 

The reductive argument, in intuitive form, is as follows; the Huffman 
switching net can be regarded as defining n class J? of geometric tignr-ea 
which tire connected or ant depending on the parity of a certain set, the 
set switches in “on” position. Wo thus tee how a perception far on one 
set, fl, can be used us a perceprrnn for j, on a second set J? . As a per- 
CCpttim for it must he of order at least |ff|. Thus the Older of i, x „ 
must he of order | fi'j. We shall use the collapsing theorem io formalize 
thia argument. But before doing so we note that a certain price has b*eu 
paid for its intuitive simplicity: the set ft is much hiegcr than the sat R', 
m fact | J?| must he of the order of magnitude of £ |ff \ so that the best result 
ten he obtained from the construction is that the order of *„ r , must increase 
wicii R rii<c log]fl|- This give* a weaker boundh tog|A| competed with 
| R ! ' \ if IV t: wish to estimate the order. 

Coonec£eaif>' on a TdftHda/ Space |Jif|. Our earliest attempts to prove 
that has unbounded order led to the following curious result; The 

predicate on an 'An X G toraidafly connected ipticc |fi| has order 

£• w. The proof is by construction: consider the space 



bi which the edges C,c and fj are Identified. Consider the family of subset?; 
of R that satisfy the conditions; 

(S3 All the shaded points belong to each X £E 

(it) For each X £ J? 1 and each i, either both points marked d 4 * 1 oi both 
points h"-' arc in ■ &*, but no other combinations arc allowed. 

Then at can be seen, for each X £ that. X is either one connected figure- 
Ot X divides, into two separate connected CigurES. Which case actually 
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occurs depend* only on the parity of \\ i|a*' £ X f| Then using the Co L- 
Japsing Theorem and the order ^ |^l theorem, w* find that ^ 

has order | J?|/12. 

The idea of this proof came from the Attempt to reduce connectivity to 
polity directJy hy representing the switching diagram: 




-— \ a . 

*%. 1 

'V - 

~.vv 

* —■ - 

,3 


J, up" 

"dtiwn'' 

“up" 


If an even number of switches are in the “down” position then a is cohumcIhI 
In □' and b to b'. If the number of down switenea is odd, 0 is connected to 
b and a' to fr. This diagram can he drawn in the plane by bringing the 
vertical connection* around the end; then one finds that the predicate 
fd is Connected to o'") has tor order some constant multiple of |K| P ]f we 
p^t the toroidal topology on fJ h the order become £ constant times |fi|; 
this is aJsn true for a U-dimensinnaJ non toroidal R. Because of those resuLts, 
we conclude that the order - obtained for <f >ifl too low, 

Auoep ]N fBuoK: We have since shown that, the order is at lease | R | w 
m the piane- 

-Sp™ 1 Other Geometrical Predicates. A number of other important geometric 
predicates that almost certainly have unbounded orders are: 

1. Symmetry; is a symmetric about some Line in 1,hc plane. ]° 

2. "Twins"; | X consists of two disjoint congruent subfigures.”| 

■J. Concentricity; \~X contains an interior hole. - ] 

Curiously enough, the predicate 

X has a single connected comparent - ] V contains a hole - ] 

has order 2. This can be shown by u conEtruction using the Euler relation, 
(Holes - 1 + Edges — Vertices — Faces), even though each separately has 
unbounded order¬ 
s'll- toniectielly and serial contpUlaiioD. It seems intuitively dear that 
the Tcason that the Abstract quality of connectivity cannot be captured 
by a machine of finite order is that it has an inherently serial character; 
one cannot conclude that a figure is connected by any simple order- 

^BuL, if « poM-iatar Jiis liirn it choBM] in signet, Mity wtltr 2 ij. required! 
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independent combination of simple tests. The *nme is true for the much 
simpki- property of parity. In the cat* of parity, there is a stark contrast 
between out "worst possible” reEuli for finite-order machines (ffllj and 
the following best possible" result for the seno! computation of parity. 
Let x,. * E , - be any enumeration of the points of if and consider the 
following algorithm, for determining the parity of |X|; 


START: 

set i loO 

EVEN: 

add I to l 

ID - | R | then STOP; parity is EVEN 

I f *i “ 0, go to EV r EN ; otherwise go to OD D: 

ODD. 

add 1 to ^ 


If J ** | R | then STOP; parity is ODD 

1J r,- = go to ODD; otherwise go to EVEN; 


where "go to o” means continue the algorithm at the instruction whose 
name is mj 

Now this program is "minimal” in two respects: first in the number of 
computation-steps per point, but more significant, in the fact that the 
program requires no temporary storage-place for partial information ac¬ 
cumulated during the computation, other than that required for the 
(Enumeration variable i. iln ft sense, the process requires one binary-digit 
of current information, but this can be absorhed fthovoj into the 
algorithm -structure.) 

I his suggests that it might be illuminating. to ask for connectivity; 
how much storage is required by the best serial ALgorithm? The answer, ft* 
shown below, is thftt it requires no more than about 2 time* that for storing 
the enumeration variable alone! To study this problem it emits that the 
Turing machine framework is the simplest, and most natural, henftusw of 
its simple uniform way of handling information storage,. 

4 ierjor Algorii'Jufi fijf CortnectiLily, Connectivity of ft geometric figure 
X is characterised by the fact that between any path (p,^> of points of 
X there is a path that Lie* entirely in X. An equivalent definition, using 
Ihe enumeration #],■■ ■, i e| of the points of R is: X i* connected if and 
only if for each point after the first point in X , there is a path to some 
*j-in X for which j! > j, fFroof: by recursion, then, each point of X ia con¬ 
nected ip the fust point in .¥.) Using this definition of connectivity ur enn 
describe a beautiful algorithm to test whether X is connected, W e witl con¬ 
sider only figures that are '"reasonably regular”—to be precise, we suppose 
that X is bounded by a number of oriented, simple, closed curves so thftt 
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For each point x, on a boundary there is dlehned a unique "next point” 
x„. on that boundary- Wc choose x,- to bo tlsu boundary point to the left of 
when facing- the complement of X. We will also assume that p-...iin> x. 
and x MS that are «NisecwfiW in the enumeration arc cidjciccni in ft. Finally, 
we will assume that X does not touch the edges of the space R. 

START. 

Set i tofl and go to SEARCH 

SEARCH: 

Add 1 to j. If i — | Fi\ „ Stop and print “Xis NULL." 

If x, g X then go to SCAN, otherwise go to SEARCH. 

SCAN: 

Add 1 to J. 3 ff = | ff |, Stop and print “ X is ccnncciga'.” 

If x, L ^ A' or x, ^ A go to SCAN, otherwise 

Set j to i and go to TR ACE, 

TRACE: 

Set j to jf * 

If j — i r Stop and print ""X is discounts fed " 

If/ > i, go to TRACE. 

If / < i, go to SCAN. 


Notice tha.L .it any point Jo the computation, it is necessary to keep track 
of the Indexes of juet the two pomes x, and x,. 

A rj,j i'vsl.v. SEARCH simply finds the first point of X in thi- ftnunieratlfl® 
of R. Once such a point of X is found. SCAN searches through aiL of R, 
eventually test in g every point of X, The current point, x, h of SCAN is 
tested follows: If i, is not in X, then mi i.rst. is m-cessary and SCAN goes 
on to Jtj. |. If the previous point x, . j was in X l.and, by induction, La pre&umed 
to have passed the test! then Xj, if in X r is connected to x,_i by adjacency. 
Finally, if je,^ X ;mrl i Jf K then x, 14 on ft boundary curve B. TBAIIB 
ciicumnavigates this boundary curve. Now if B is a boundary curve it is 
either lij an exterior boundary' of a previously encountered Component 
of X\ in which cssc some point of fi must have been encountered before 
or (itj B is an interior boundary curve, in which case a point of ft must 
have been encountered before reaching x,_■ which it inside B or (iii) B is 
the exterior boundary curve of a never-before-encountered component, of 
A", the only case in which TRACE will return to x, without meeting an x, 
for which J < i. Thus SCAN will rum up to J = | if and only if X has a 
single nonempty connected component. 

Note that we can cvwnf the number of component* of X by introducing 
K, Initially zero, And adding 1 to K each time TKACH reaches tins i = j 
exit. Note also that- the algorithm is quite efficient; the only points examined 
more them once are gome of the boundary points, and none of them is 
examined more than three times {see figure bt-lowi, 
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TV Turing Afachirw Version of Ih* Coniwcfujifr .4 iganthm . It. ts convenient 
to assume that Tf is a 2' X T square array. Lei x h - - x riM . U an enumera¬ 
tion of the points of R in the order 

1 r+1 , (r-l)2 n +l 

2 2 '+ 2 C 2 "-l)r +2 


2 * 2 - 2' . j . 2" - 2". 

Tltis choice of dmiemkm and enumeration makes available a simple way 
to reprewht Lhc situation to □ Turing machine. The 'I'uriitK machine must 
ho able to specify a point x ( of ft, find whether a,e X r and in Case *, is a 
boundary point of X, hud the index ;* of the "left neighbor" of je,. The 
Turing Machine tape will have the form 


h 

• ■ Jl • ^ 

I. 

, r a , . 

J, 

*■ * ft ■ ■ 


- , n - r 

K 


where Ll ’ > n ■ ■" denotes an interval of n blank square Then the intervale 
to the right of /, and / ^ can hold the x and y coor-dinates of a point of fi. 
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We will suppose that the Turing machine :s coupled with the outside 
world, i.e, r the figure X r through mu “oracle" that works as follows: certain 
internal states of the machine have the properly fhat when entered, the 
resulting neit slate depends on whether the coordinates in the / {or J) 
intervals designate a point in X. ft can he verified, though the details 
ate tedious, that nil the upcraiionE described in the algorithm can be 
paftHMd by a Jsxcd Turing machine that, uses no tape squares other than 
those in "- rrt . intervals, for example, "i = j Ji21” if and only if there 
are all zeros in the " r ■ ft ■ -"s following i, and 1^. "Add 1 to i n is equivalent 
W- "start at and move left, changing l's to O's until a 0 is encountered 
and ebanged to 1 or until Jj. is met. The only nontrivial operation is com¬ 
puting j* given j. But this requires only examining the neighbor* c>f je„ 
Ood that Ls done by adding ± 1 to the J, end J y coordinates, and consulting 
the oracle, 

Since the Turing machine can keep track of which interval 

it is in, we really need only one symbol for punctuation, so the Turing 
machine can be a 3-symbul machine- By using a block encoding, one can 
u&a a 2-symboi machine, and, omitting details, we obtain the result; 

Theorem. Far any t there es a 2 -symbol Turin# machine that cun verify 
the connect juity af a future X on any rectangular array R, using less than 
(2 +*}!og. : |ft| mjunres af tape. 

For comeiily there is a similar procedure that makes three tests; 

1, X is not disconnected by any vertical line that does not intersect. X, 

ji. The intersection of X with any vertical line is a connected segment. 

iii. The outer boundary of X does not change the sign of its curvature 

A detailed construction shows that each test requires only one judex 
point, $0 that 

TlJEOR&M. Far any t there is a 2-symbol Turin# machine 4&s( can verify 
th# fflnwtUy of fl figure X on- any rectangular array R, using less than 
{1 +■ t) log.| of tape. 

This last result is certainly minimal since log, Jl squares are neetltd just 
to indicate a point of ft, and all points must be examined. We are quite 
sure that the connectivity algorithm is minimal, also, in its list of tape, 
but wc have no proof. In fact, we do not knew any method, in general, 
to show that an algorithm ls minimal in storage, except when information- 
theoFEtic arguments can he used, Incidentally, it is not hard to show that 
j |X| is prime - ! requires no more than {£ +- 1 ) liyjjl ft| squares janci pre¬ 
sumably needs more; than (2 - *) Jog 3 | R |). 

We do not definitely know any guanine predicates that require higher 
orders of storage, but we suspect that in an appropriate sense, the topolemical 


LfNEARLY- UNRECOGNIZABLE PATTEflNS 


st i 

equivalence of two figures (e.g., two components of X) require® something 
mftt'* Like |fij than like Lugl^l squares. There are„ of tourne, recursive 
fufKtion-thwpetic predicates that require arbitrarily high, indeed non- 
eomputablfip orders of storage, but none of that is known to have straight- 
iorward geometric interprets lions. 

VIII,. M ulci-1 aj er perccptionR. We have found a number of limitation^ 
of pcreoptroriR, as defined above, and w* have suggested that these may 
point toward as yet unknown theorems about parallel machines in general. 
On the other hand one suspect* that some, al least, of the results above 
arc not ao general, and might not survive m inor raisjunlionE of the definitions. 
One direction of generalization that stems important is that of relaxing 
the constraint that the *' 5 bo simply weighted and added. W* have not 
found, any particularly enlightening generalization on the lowest level—c,g., 
of replacing addition by an arbitrary commutative operation. An easier 
direction it to consider compositions of pcrceptrons. The remainder of 
this section explore!; some kinds of composite perceptions. LfnforUanstely 
we dp not understand thorn very well, so this section is more concerned 
with problem-posing than with probJem.-solving 
Giirnfw Afack inca-, Consider functions of the form 

P S-djf P *i ^ f/l > i ""] ^ 

/ I 

This form was proposed and realized in a Mobs of machines built by A. 
Camba \2 . It is essentially an order-] composition of order-1 JMTCBpIrons, 
and is yf interest to us far a number of reasons: 

(i) The parity problem is solved neatly by 

p£<- UTZ*i>i'l >o"|. 

>-* hJ : 

Thtis only |if| functions are needed, each itself of order 1 in the | je,J. In 
fact , any predicate/(| Jf|) that depends; oniy on the area fXI cam be 
icutizedi. as 

/tl-STf) -fffl + X P I>;> J'l - (fij+ 1) -/{»K 

j-D 

r The prublcm that Jed t* out formulation of the AND-OR fbeanm aUo 
is solved neatly: 

rr x>- £*>mpx*- x»oi *21 

i& l if and only if j A fi A | > IX f\ C\ and \X n B\ > | X fl Cj, ThtB 

might suggest that thia class of machines might transcend the other kind* 
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of limitations we have found fur machines of finite-order,* 

Wc are quite cetunn that this impression is misleading; that the deeper 
geometric properties She still outride the teach of this kind of "2-J*yer” 
percept™. The inclosion of AND and OR is due to the 2-Uycr construction; 
any Boolean Junction is obtainable, in such a manner, through its normal 
fdrm. but for most functions there will atilt he too many terms for practical 
interest. The X fl A | > | X n Cj ~| type of predicates are within reach 
bKauiie they are simple area functions And hence Jit. precisely the inner 
first-order predicate forms. In fact, any dfisaJ^f figure* can be recognised 
by a Gemha-machine because 

I X rx x < * I 

11 JfF J? jfgJC 

realizes it. Rut, this general form requireE a special u ' G-am ha - musk 13 for 
each XG.p 1 . Although the above examples show that in special esses 
more economical representations are possible, this is not true in genera] 
tas one can ace by considering the number 2 1 " of possible functions), In 
particular we conjecture, for example, that for the Connectivity Predicate, 
the machine would require a number of masks of an order approaching 
the number of simple-dosed-curves in ft, Even for convexity, we doubt 
that that predicate can be realized with significantly fewer than the number 
oi t s needed for the order-S 1-layer machine. 

(ii.l In spite oi its apparent simplicity, analysis of the geometric predicate 
problem for Gumba machines appears to require methods quite different 
trcuu tr.ose we have used. First, because of the arbitrary order-T predicate 
permitted in. the inner sum, the notion of order dues not seem to apply, 
and theorems must concern restrictions on the numbers of terms. Second, 
we have not found a way to curry the group-averaging methods into the 
inner * M coefficients, so that we cannot use the techniques that Cume from 
the grpup-invariance theorem. It in difficult to see how to analyse other 
multi-layer and composite perceptrons until this simple vase is better 
understood. How much weaker are the machines with > 0 or these 
with all #= 07 W® have no characterization of what they can da. 

i.iii) The G am ha-machine is of considerable practical interest because 
of the possibility of retiring the inner, and even the outer, sums by in- 
eipehEive, highly parallel optical methods. Using coherent Ii^tit and properly 


' Pf(J4* that Ihc (iimhu-iuatliiuie can have ntder Hv laj^b- bi \R , in ik#- |. Lf tine injur 
tredicaW threshold v*er* jcmnvrd then, became 

xj, 

(ms would have mv.r*l> an ureter-L function. ui she Jjrjf. 
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pt-efWirisd photographic transparencies, one can realise each inner sum (even 
with complex CMfflcLentaE) with a picture p, whose density at point- *, is 
By shrewd optics, one can even do this (with fined p,) for aLI traumatic™ 
of the source pattern X. Because of these technological possibilities it U 
important to have a better theory; we aspect that the resuLr. will be lavorable 
Co problems Dike recognition of printed characters, but still very peor for 
the more abstract properties like detection of connectivity, symmetry! 
topological equivalence, and the like. 

IX. The diftBieterdimilcd pcrccptrort. In this section we discuss the power 
and limitations of the '"rbameter-limited" perceptions: those in which 
each ^ tan see only a circumweriirtd portion of the retina R. 

We consider 11 machine that sums the weighted evidence about a picture 
obtained hy experiments d, each of which report on the state of affairs 
within u circumscribed region r, of .iwi^jetcr I'-caa Hum $r equal to some, length 
D. That is, Diameter (&{$)) ■*£ I). We will suppose that f? >s uniform over 
the of the machme (each actual region that affects a <t, can be smaller, 
but not larger}. We suppose also that in a practical sense D is small com- 
piired with the full dimensions of the space fi, That is, D should be small 
enough tliat none ol the a's can see the whole of an interesting figure (t>r 
else we would not have an effective limited-ufiHmeler situation, and there 
would he no interesting theory ) but D should he Urge enough that a <p, has 
a chance to detect an interesting '"local feature” of the figure. 

We will consider first some things that a diameter-limited perception 
can recognise, and then some of the things it cannot. 

fill 1 dlorrlr picture, or- blitck picture. A tliamstEr-limited p-vrecpLror. can 
tell when m picture l.a entirely black, or entirely whits; suppose that Ihe 
set ol d/s is chosen to cover the retina in regions:, that may overlap, and 
that we define p, tp be zero when aLL the points it can see are white, other¬ 
wise its value is 1. Then J>, > !) if the picture has one or more black points, 
and not if the picture is blank- SbmilarLy, we could define the *,'* to be 1 
when they see any while point, fl otherwise, thus distinguishing the alL- 
bl&ek picture from all others. 

For Later examples., it is important here to notice why these pattern.* 
can be recoinized: it is not that any fc-unit can really say that there- is 
strong evidence th-im the figure is all-white (although it has a slight cor¬ 
relation with thisf; but any can definitely say that it has conclusive 
evidence that the picture Ls not nil white. Boms interesting patterns- h-flve 
this character; that one can reject all pictures not in the class because each 
must- have, somewhere or other, a local feature that Is definitive and can 
be detected by what happens within a region of diameter D. 

(b] Area ciiZs, We can distinguish, for any number.S, the class of figures 
whose area is greater than 5. To do thift »c define a #,- for each point to 
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be . if that point is bLaek, 0 otherwise. Then ^ jc, !> S i_> □ recognizer for 
the dais in question. (One «ui do slightly better; if the #'a look at region* 
Of WM A, then one tan recognise this pattern by using only of the order 
of (/f Log A)/A units, | 

(*) Nonintersecting lints. One can say that a pattern is composed of 
nonmutweting lines if, [n each small region t the pattern u competed of 
separateline-scgmenta, or blank, Then, if we make each # have value zero 
wh™ this condition is met, unity when it is not, then JT>, > 0 will reject 
all figures not in the class. 

(d) T , H tnicies. W e can make a diameter-limited perception recoEnize 
the figures consisting of eiactJy one triangle (cither solid or outline) by 
the folkwiDf (nek: We use two kind* of **« the first baa weight + 1 if 
it* field contains « vertex (two line segment.-? meeting at an angle), other¬ 
wise it*, value is zero, The second kind, d, T . has value zero if it* field is 
blank, or contains a line segment, solid black area, or a vertex, hut has 
Vftluo + 1 if the field contains anything else, including the end of a line 
segment. Provide enough of these p's go that the- entire retina is covered, 
in Eionoverlappmg fashion, hy both types. Finally assign weight 1 to the 
brat type and a very largo positive weight IV to those of the second type. 
Then 

Z*i - wz>‘ < * 

will be a specific recognizer for triangles, {But also the null-pfctun L s 
accepted,) Similarly, by requiring that the first kind of unit recognize righl- 
angis vertices, the machine can he made to recognize the daax of rectangles 
(setting the threshold to be <5>. 

Note that this does not generalize to a very wide class of geometric 
recognition abilities, The triangle and rectangle case* are rather peculiar;, 
the triangle because it is the simplest figype that has true vertices. The 
rectangle cun be recognized because it has four equal angles; the system 
Cannot he specialized to recognize, for example, exactly the squares. It 
ls interesting, in view of the limitations we will establish shortly, to aee 
why the** patterns can be recognized by the diameter-limited machine; 
a rectangle is the only figure that has four or }euxr right angles and no 
free Line ends* ate. 

l.ej Afisofttfe temptlate-mQiching, Suppose that one want*, the machine to 
recognize exactly a certain figure .Xf, and no other, Then the diameter- 
Limited machine can be made to do this by partitioning the retina into 
tug ions, and in each region a £-function hat a value 0 iUT that part of the 
retina is exactly matched to the corresponding part of X Ul otherwise the 
value is 1. Then 

Z*i < I 

if and only Lf the picture is exactly X 6 . 
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Ncte however, that this scheme worts just on a particular object in a 
particular PQsUiom it cannot he generalised to recognize a particular object 
in any position (or even, in (finer a], in two positions), In fact we show 
m the wt »ctmn that even the simplest possible figure, namely one that 
eonsjstsof jo&t one point, cannot be recognized independently of positicra! 

Qmaxtiy. The remarks in f&, Example c, footnote apply to the 
diameter-Umited case. 

£mitoxw« 0 f ammeter.tinted P&ctptrm, Now we consider ^me of 
the basic limitation* of the diameter-limited perception, by exhibiting and 
analysing some patterns they cannot recogniie. 

Ig) The figure containing m® single black point. This is the fundamental 
crjunter-ejiumpie. We want a machine 

U ana I, but reject figures with ar^ 0 Qr area greater 

an , Clearly this can be defined by two area cuts |i.e., area >0 AND 

area <2), but it cannot be realized by a Linear threshold function with 
the area-restnetton. 

To see that this cannot be done, Hippos* that f*f, |*.j An dfl have been 
selected. Present first the blank picture, X t . Then, defining /(X) = £ nj *(x) 

we have f{X L ) <. f. Now present a figure, X n , containing only one point x, 
Vte must then have 


fix ,3 as. 

The change in the sum must he due to a change in the values of some of 
thetf S- In tact, si must be due to changes only in <S*e for which ft SU) 
sjnee noilsing etae in the picture! has changed. In, any case, 

Now choose another point i 2 which is father than D away from *, Then 
no SM can contain both *, and * t . For the figure X, containing only * 4 
we must 41I&0 have 

W fW •=£<"****• 

Now consider the figure X J2 containing both at, and jc 2 . The addition, to 

7' at puint *' Cfln affECt only *'* for which igSi*), and these are 
ohanued exactly us they are chatted when the aLL-blank picture X, is 
changed to the picture X L . Therefore 

ftx jJ ) + jw p - tixj} 

and by (1? assd (2), 


fix ls t > p. 
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but ws require that 

/( Xff} £ Cl. 

REJaakll. Of course, this is the same phenomenon noted in the introduc¬ 
tion bo $JL 

(hi Aren ‘tegme.ntet The diameter-limited peuaplrtm cannot recognize 
the cia&s of figures whoso arcus zt lie between two bounds j 4 , i A £ 4 a . 

PluiUr. This foLLows from (.he method of (aj above, which is a special 
case of Iras, with Ay — 1 and A% = I. But using the method of £l r Example 
tvii], this, recognition ts possible with order 2 if the [linmeter^linutatioD 
is reLsied. 

fi) nfsi, The diameter-limited percept-non cannot decide when 

the picture is a dijgle K connected whole, as distinguished from two or more 
disconnected pieces. 

Proof. Consider the four pictures 
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and suppose that the diameter £J is of the order indicated by the dotted 
circle. Now figures Xn, and X m are Connected, but and ,V 3: ane dis¬ 
connected. Suppose that there were a sen of ^’s and n’s and a such that 




T*.a,a,-f.Jfnil ^ 0, 
£«i*.-(Xin) £ (f t 




so that those four figures wm- correctly separated. But them just as in 
the previous argument we would have for all <?,, 

^itXyy} nr 


because the two changing regions are more than D apart, hence 

;^5 + 0 — 0 = P 


contradicting the separation requirement. 
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.4fkbrn*.krt|iFem^wE: Wl* would lake- To thank many sturtfcriti; and aMociates. 
purtkularly William Hemne-man, Dona Slians* asul Jwhn White. Jor their 
tuniributions to this woik. 
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